Randomized Iterative Hard Thresholding: A Fast Approximate MMSE Estimator for Sparse Approximations

Typical greedy algorithms for sparse reconstruction problems, such as orthogonal matching pursuit and iterative thresholding, seek strictly sparse solutions. Recent work in the literature suggests that given a priori knowledge of the distribution of the sparse signal coefficients, better results can be obtained by a weighted averaging of several sparse solutions. Such a combination of solutions, while not strictly sparse, approximates an MMSE estimator and can outperform strictly sparse solvers in terms of mean l reconstruction error. Existing algorithms show promising results in improving performance based on approximate MMSE estimation, but can be prohibitively expensive for large-scale problems. We introduce a novel method for obtaining such an approximate MMSE estimator by replacing the deterministic thresholding operator of Iterative Hard Thresholding with a randomized version. This algorithm achieves the performance of the recently introduced RandOMP with much greater computational efficiency, suitable for application to largescale problems.